Á¤º¸Ã³¸®ÇÐȸ ³í¹®Áö ¼ÒÇÁÆ®¿þ¾î ¹× µ¥ÀÌÅÍ °øÇÐ
ÇѱÛÁ¦¸ñ(Korean Title) |
InfiniBand RDMA Åë½ÅÀ» À§ÇÑ Apache StormÀÇ À籸¼º |
¿µ¹®Á¦¸ñ(English Title) |
Reconfiguration of Apache Storm for InfiniBand Communications |
ÀúÀÚ(Author) |
¾ç¼®¿ì
¼Õ½Ã¿î
¹®¾ç¼¼
Seokwoo Yang
Siwoon Son
Yang-Sae Moon
|
¿ø¹®¼ö·Ïó(Citation) |
VOL 07 NO. 08 PP. 0297 ~ 0306 (2018. 08) |
Çѱ۳»¿ë (Korean Abstract) |
º» ³í¹®¿¡¼´Â ºÐ»ê ½ºÆ®¸² ó¸® ÇÁ·¹ÀÓ¿öÅ©ÀÎ Apache StormÀ» °í¼º´É Åë½Å ÀåºñÀÎ InfiniBand¿¡ Àû¿ëÇÏ´Â ¹æ¾ÈÀ» ´Ù·é´Ù. InfiniBand »ó¿¡¼ StormÀ» µ¿ÀÛ½ÃÅ°´Â ½¬¿î ¹æ¹ýÀº IPoIB (IP over InfiniBand)¸¦ »ç¿ëÇÏ´Â °ÍÀÌ´Ù. ±×·¯³ª ÀÌ ¹æ¹ýÀº ³ëµå¿¡ ½É°¢ÇÑ CPU ºÎÇϸ¦ ¹ß»ý½ÃÅ°´Âµ¥, ÀÌ´Â ÀæÀº ¹®¸Æ Àüȯ°ú ¹öÆÛ º¹»ç¿¡¼ ±âÀÎÇÏ´Â °ÍÀ¸·Î ³ªÅ¸³µ´Ù. À̸¦ ÇØ°áÇϱâ À§ÇØ, Storm¿¡¼ InfiniBandÀÇ RDMA (Remote Direct Memory Access) ±â´ÉÀ» »ç¿ëÇÏ´Â »õ·Î¿î Åë½Å ¹æ½ÄÀ» Á¦¾ÈÇÑ´Ù. ù°, Storm¿¡¼ RDMA ±â´ÉÀ» ÀÌ¿ëÇϱâ À§ÇØ, ±âÁ¸ Åë½Å ÇÁ·¹ÀÓ¿öÅ©ÀÎ Netty¸¦ ´ëüÇÏ´Â »õ·Î¿î ÇÁ·¹ÀÓ¿öÅ©ÀÎ RJ-Netty (RDMA/JXIO Netty)¸¦ ¼³°è ¹× ±¸ÇöÇÑ´Ù. µÑ°, StormÀÌ ±âÁ¸ Netty¿Í RJ-Netty¸¦ ¸ðµÎ »ç¿ëÇÒ ¼ö ÀÖµµ·Ï °ü·Ã Ŭ·¡½ºµéÀ» °³¼±ÇÑ´Ù. ¼Â°, RJ-NettyÀÇ ¼º´ÉÀ» ÃÖ´ëÈÇϱâ À§ÇØ ¸ÖƼ½º·¹µå¸¦ Áö¿øÇϵµ·Ï JXIO ¼¹ö ±â´ÉÀ» °³¼±ÇÑ´Ù. ½ÇÇè °á°ú, Á¦¾ÈÇÑ RJ-Netty´Â EthernetÀº ¹°·Ð IPoIB¿¡ ºñÇؼ ¸Þ½ÃÁö 󸮷®À» Çâ»ó½ÃÅ°¸é¼µµ CPU ºÎÇϸ¦ Å©°Ô ÁÙÀÎ °ÍÀ¸·Î ³ªÅ¸³µ´Ù. º» ³í¹®Àº Apache StormÀ» InfiniBand »ó¿¡¼ µ¿ÀÛ½ÃŲ ÃÖÃÊÀÇ ½Ãµµ·Î, °í¼º´ÉÀÇ InfiniBand RDMA¸¦ »ç¿ëÇÏ¿© StormÀÇ Ã³¸® ¼º´ÉÀ» Çâ»ó½ÃŲ ¿ì¼öÇÑ ¿¬±¸ °á°ú¶ó »ç·áµÈ´Ù.
|
¿µ¹®³»¿ë (English Abstract) |
In this paper, we address how to apply Apache Storm, a distributed stream processing framework, to InfiniBand, a high performance communication device. An easy way to run Storm on InfiniBand is to simply use IPoIP (IP over InfiniBand). However, this method causes a serious CPU load on the node, which is caused by frequent context switches and buffer copies. To solve this problem, we propose a new communication method using InfiniBand's Remote Direct Memory Access (RDMA) function in Storm. First, we design and implement RJ-Netty (RDMA/JXIO Netty), a new framework that replaces Netty, the legacy framework, to exploit RDMA functionality. Second, we reimplement the related classes so that Storm can use both existing Netty and new RJ-Netty. Third, we extend the JXIO server functionality so as to support multi-threading to maximize the performance of RJ-Netty. Experimental results show that the proposed RJ-Netty significantly reduces CPU load while improving message throughput compared to IPoIB as well as Ethernet. This paper is the first attempt to run Apache Storm on InfiniBand, and we believe that it is an excellent research result that improves the performance of Storm by using InfiniBand RDMA.
|
Å°¿öµå(Keyword) |
Apache Storm
InfiniBand
IPoIB
RDMA
Netty
RJ-Netty
|
ÆÄÀÏ÷ºÎ |
PDF ´Ù¿î·Îµå
|